Vision-Speech System Becoming Efficient and Friendly through Experience

This paper presents a vision-speech system for service robots that can learn the user’s customs and objects fixed in the environment while helping the user, and can perform their tasks more efficiently with less user’s burden. We are working on a service robot that brings objects ordered by the user through speech. The robot needs vision to recognize the objects. It asks the user for help by speech if its vision fails. In early stages, it asks the user for help many times and vision takes time to detect the objects. In later stages, however, the user does not need to say details because the robot knows where the objects usually are through experience. Moreover, the vision processing time is greatly reduced because it knows what operations can work in such cases. Experiments using a robot system show the usefulness of the proposed system.